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A statement identifying the real party in interest is contained in the brief 

(2) Related Appeals and Interferences 

A statement identifying the related appeals and interferences which will directly 
affect or be directly affected by or have a bearing on the decision in the pending appeal is 
contained in the brief 

(3) Status of Claims 

The statement of the status of the claims contained in the brief is correct. 

(4) Status of Amendments After Final 

The appellant's statement of the status of amendments after final rejection 
contained in the brief is correct. 

(5) Summary of Invention 

The summary of invention contained in the brief is correct. 

(6) Issues 



The appellant's statement of the issues in the brief is incorrect. The examiner 




withdrawn the rejection of claims 5, 7, 8, 10, 31, and 38 responsive to the appellant's 

brief The following claims 1, 2, 3, 4, 6, 1 1, 32, 33, 34, and 35 APrrejected under 35 

A 

U.S.C. 102(b) as being anticipate by Booth et al. (USP 5,922,079), 

(7) Grouping of Claims 

The rejection of claims 1, 2, 3, 4, 6, 1 1, 32, 33, 34, and 35 stand or fall together 
because appellant's brief does not include a statement that this grouping of claims does 
not stand or fall together and reasons in support thereof See 37 CFR 1. 192(c)(7). 

(8) Claims Appealed 

The copy of the appealed claims contained in the Appendix to the brief is correct. 



Application/Control Number: 10/053,748 Page 3 

Art Unit: 2863 

(9) Prior Art of Record 

5,922,079 Booth et al. 07-1999 

(10) Grounds of Rejection 

The following ground(s) of rejection are applicable to the appealed claims: 

Claim Rejections - 35 USC § 102 

1 . The following is a quotation of the appropriate paragraphs of 35 
U.S.C. 102 that form the basis for the rejections under this section made in this 
Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this 
or a foreign country or in public use or on sale in this country, more than 
one year prior to the date of application for patent in the United States. 

2. Claims 1-4, 6, 1 1 , and 32-35 are rejected under 35 U.S.C. 102(b) as being 
anticipate by Booth et al. (USP 5,922,079). 

Regarding claim 1, Booth et al. teach an automated analysis and 
troubleshooting system is provided that identifies potential problems with the test 
suite, and also identifies probable modeling errors based on incorrect diagnoses 
(e.g. Col. 5, lines 35-40), the method comprising step of evaluating a diagnostic 
efficacy of the test suite (e.g. Col.9, lines 14-15) using a probability of one or both 
of a correct diagnosis and incorrect diagnosis by the test suite (e.g. Col.1 1 , lines 
15-18, lines 27-29). 

Regarding claim 32, Booth et al. teach a test system that that identifies 
potential problems with the test suite, and also identifies probable modeling 
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errors based on incorrect diagnoses comprising: a processor (e.g. CoL6, lines 
66-67); a memory (e.g. CoL7, lines 52-55); and a computer program stored in the 
memory and executed by the processor, wherein the computer program 
comprises instructions that, when executed by the processor (e.g. Col.6, lines 
61-65, Col.7, lines 10-18), implement evaluating the test suite (e.g. Col. 9, lines 
14-15) using a probability of one or both of a correct diagnosis and incorrect 
diagnosis to determine the efficacy (e.g. Col.11, lines 15-18, lines 27-29). 

Regarding claims 2, 33, Booth et al, teach the evaluation comprises 
suggesting a test to add to the test suite to adjust an overall test coverage of the 
test suite (e.g. Col.6, lines 38-45, Col.11, lines 29-35). 

Regarding claims 3, 35, Booth et al. teach suggesting a test comprises: 
creating a simulation database 124 of the test suite; determining a probability of a 
correct diagnosis (e.g. Col.6, lines 49-54) and a probability of an incorrect 
diagnosis for the test suite using the database (e.g. Col.9, lines 33-61); and 
creating a list of suggested tests from the determined probabilities (e.g. Col. 10, 
line66-Col.11, line 35). 

Regarding claim 4, Booth et al. teach each suggested test on the list 
comprises test coverage (e.g. Col.1 1 , lines 29-31 ). 

Regarding claim 6, Booth et al. teach identifying a test to delete from the 
test suite (e.g. Col.1 0, line 66-Col.l 1, line 15), the deletable test having a 
minimal effect on an overall diagnostic efficacy of the test suite (e.g., if an 
incorrect diagnosis is made, the automated analysis system identifies ways of 
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changing the rank order of diagnoses, including coverages that can be reduced 
and identification of operation violations that can be eliminated, Abstract). 

Regarding claims 1 1 , 34, Booth et al. teach the method of evaluating a 
diagnostic efficacy of the test suite using a probability of a diagnosis (e.g. Col.1 1 . 
lines 24-64); creating a simulation database 124 of the test suite (e.g. Col. 6, lines 
49-54); determining a probability of a correct diagnosis and a probability of an 
incorrect diagnosis for the test suite using the database (e.g. Col.1 1 , lines 1 5-18, 
lines 27-29); using the determined a probability to evaluate the test suite (e.g. 
CoL9, lines 14-15). 

Allowable Subject Matter 

3. Claims 5, 7-10, 12-31, 36-37 are objected to as being dependent upon a 
rejected base claim, but would be allowable if rewritten in independent form 
including all of the limitations of the base claim and any intervening claims. 

The following is a statement of reasons for the indication of allowable 
subject matter: 

Regarding claim 5, none of the prior art of record teaches or suggests the 
combination of a method of determining a revision of a test suite of a model- 
based diagnostic testing system, wherein the method comprising step of 
evaluating a diagnostic efficacy of the test suite using a probability of one or both 
of correct diagnosis and incorrect diagnosis by the test suite, wherein the 
evaluation comprises: suggesting a test to add to the test suite to adjust an 
overall test coverage of the test suite, wherein suggesting a test comprises: 
creating a simulation database of the test suite; determining a probability of a 
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correct diagnosis and a probability of an incorrect diagnosis for the test suite 
using the database; and creating a list of suggested tests from the determined 
probabilities; wherein the evaluation further comprises: identifying a test to delete 
from the test suite that comprises: determining a probability of a correct diagnosis 
for a modified test suite using the database, the modified test suite having a 
selected test removed from the test suite; computing an efficacy value associated 
with the selected test using the determined probabilities of a correct diagnosis for 
the test suite and the modified test suite; and generating a list of deletable tests 
and associated efficacy values. It is these limitations as they are claimed in the 
combination with other limitations of claim, which have not been found, taught or 
suggested in the prior art of record, that make these claims allowable over the 
prior art. 

Regarding claim 7, none of the prior art of record teaches or suggests the 
combination of a method of determining a revision of a test suite of a model- 
based diagnostic testing system, wherein the method comprising step of 
evaluating a diagnostic efficacy of the test suite using a probability of one or both 
of correct diagnosis and incorrect diagnosis by the test suite, wherein the 
evaluation comprises identifying a test to delete from the test suite, the deletable 
test having a minimal effect on an overall diagnostic efficacy of the test suite, 
wherein identifying a test comprises: aeating a simulation database of the test 
suite; determining a probability of a correct diagnosis for the test suite using the 
database; determining a probability of a correct diagnosis for a modified test suite 
using the database, wherein the modified test suite is the test suite having a 
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selected test removed; computing an efficacy value for the modified test suite 
using the determined probabilities; and generating a list of deletable tests using 
the computed efficacy values. It is these limitations as they are claimed in the 
combination with other limitations of claim, which have not been found, taught or 
suggested in the prior art of record, that make these claims allowable over the 
prior art. 

Regarding claim 12, none of the prior art of record teaches or suggests 
the combination of a method of evaluating a diagnostic efficacy of a test suite of 
a model based diagnostic testing system, wherein the method comprising step of 
creating a simulation database of the test suite, wherein creating a simulation 
database comprises step of simulating an application of the test suite to a device 
under test, the device under test comprising one or more components; and 
recording a probable result of the application in the simulation database, the 
simulation database being represented by a table having a plurality of columns 
and a plurality of rows, the plurality of columns comprising a component pattern, 
a test result pattern, and a number of occurrences, wherein the component 
pattern encodes which component is good or bad, each component of the device 
under test being represented by a unique position number within the component 
pattern, wherein the test result pattern encodes which of the tests of the test suite 
failed or passed, each test in the test suite being represented by a unique 
position within the test result pattern, wherein the number of occurrences 
represents a number of times that a given combination of the component pattern 
and the test result pattern occurred during a simulation, the number of 
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occurrences being an integer greater than or equal to zero, and wherein each 
row of the plurality of rows corresponds to a different unique pattern of good and 
bad components. It is these limitations as they are claimed in the combination 
with other limitations of claim, which have not been found, taught or suggested in 
the prior art of record, that make these claims allowable over the prior art. 

Regarding claim 31 , none of the prior art of record teaches or suggests 
the combination of a method of evaluating a diagnostic efficacy of a test suite of 
a model based diagnostic testing system, wherein the method comprising step of 
creating a simulation database of the test suite, determining a probability of one 
or both of correct diagnosis and incorrect diagnosis by the test suite using the 
database, using the determined probability to evaluate the test suite; wherein the 
created simulation database comprises a Monte Carol simulation of the device 
under test model, the database having a set of entries, each entry having a field 
for a number-of-occurrences value, a field for a test result pattern, and a field for 
a component state pattern. It is these limitations as they are claimed in the 
combination with other limitations of claim, which have not been found, taught or 
suggested in the prior art of record, that make these claims allowable over the 
prior art. 

Regarding claim 36, none of the prior art of record teaches or suggests 
the combination of a system that determines efficacy of a test suite of a model- 
based diagnostic testing system comprising: a processor; a memory; and a 
computer program stored in the memory and executed by the processor, wherein 
the computer program comprises instructions that, when executed by the 
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processor, implement evaluating the test suite using a probability of one or both 
of correct diagnosis and incorrect diagnosis by the test suite to determine the 
efficacy, wherein the instructions that evaluate the test suite further comprise: 
determining a probability of a correct diagnosis for a modified test suite using the 
database, the modified test suite having a selected test removed from the test 
suite; and wherein using the determined probability comprises: computing an 
efficacy value for the modified test suite using the determined probability of a 
correct diagnosis for both the test suite and the modified test suite; and 
generating a list of tests to delete from the test suite based on the computed 
efficacy value. It is these limitations as they are claimed in the combination with 
other limitations of claim, which have not been found, taught or suggested in the 
prior art of record, that make these claims allowable over the pnor art. 

(11) Response to Argument 

A. The appellants have argued, regarding the 35 U.S.C. 102(b) rejection 
of claims 1 , 2, 3, 4, 6, 1 1 , 32, 33, 34. and 35 as anticipated by Booth et al. (US 
5,922,079). 

The reiection of claim 1 

In response to Appellants' argument that "Booth et al. do not contain any 
teaching or suggestion evaluating a diagnostic efficacy of the test suite using a 
probability of one or both of a correct diagnosis and incorrect diagnosis by the 
test suite" (Appeal Brief, page 8, paragraph 5), the examiner disagrees. Booth et 
al. teach the test system 100, the model based diagnostic system 106, test suite 
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analysis 116 and model debug 120 are all typically computer or processor based. 
That is, software is executed by a processor within an automated test system or 
the test system is controlled by a separate computer. In general, the test suite 
analysis 116 and model debug 120 can be performed by a processor or 
computer used for the test system 100 or used for the model based diagnostic 
system 106, Alternatively, test suite analysis and model debug may be 
performed offline on a separate system or distributed among several systems. 
Note also that some of the statistical inferences involved may be determined by 
fuzzy logic or neural networks so that the word "automated" is not restricted to 
sequential execution of computer instructions (Fig.1, Col.6, line 60-Col.7, line 7). 
The test suite may be evaluated for overall accuracy by analysis of historical data 
(FIG. 1, 126). For example, the system could compute the mutual information (or 
other statistical or information theoretic measures) between the model-based 
diagnosis and the distribution of TFC's recorded in the database (Col. 9, lines 14- 
19). Booth et al. disclose a method of identifying improvements to a test suite in 
diagnosability analysis, the test suite based on a model of a system, the method 
comprising automatically identifying components, the failure of which has a 
probability. of detectability by the test suite (Col.8, lines 15-31 , Col. 13, lines 59- 
65). The debug of the model based on incorrect diagnoses (Col. 9, lines 33-34). 
Historical data may be used in conjunction with the model-based diagnostic 
system: providing cases for performance evaluation of the model-based system; 
allowing evaluation of the effect of changes suggested by debugging on overall 
performance (Col.1 1 , lines 50-60). This feature is seen to be an inherent 
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teaching evaluating a diagnostic efficacy of the test suite using a probability of 
one or both of a correct diagnosis and incorrect diagnosis by the test suite as 
intended. 

The rejection of claim 2 

In response to Appellants' argument that "Booth et al. do not contain any 
teaching or suggestion evaluating a diagnostic efficacy of the test suite using a 
probability of one or both of a correct diagnosis and incorrect diagnosis by the 
test suite" (Appeal Brief, page 10, paragraph 4), the examiner disagrees. Booth 
et al. teach an automated analysis and troubleshooting system is provided that 
identifies potential problems with the test suite (ability of the model to detect and 
discriminate among potential faults), and also identifies probable modeling errors 
based on incorrect diagnoses (CoL5, lines 36-40). Booth et al. teach step of 
evaluating a diagnostic efficacy of the test suite (e.g. Col. 9, lines 14-15) using a 
probability of one or both of a correct diagnosis and incorrect diagnosis (failure 
probability term) by the test suite (e.g. a probability of detectability by the test 
suite, Col.11, lines 15-18, lines 24-29, Col. 13, line 64). 

The reiection of claim 3 

In response to Appellants' argument that "Booth et al. do not contain any 
teaching or suggestion determining a probability of a correct diagnosis and a 
probability of an incorrect diagnosis for the test suite using the database; and 
creating a list of suggested tests from the determined probabilities" (Appeal 
Brief, page 1 1 , paragraph 5), the examiner disagrees. Booth et al. teach a test 
comprises: creating a simulation database 124 of the test suite; determining a 
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probability of a correct diagnosis (e.g. Col.6. lines 49-54) and a probability of an 
incorrect diagnosis for the test suite using the database (e.g. Col.9, lines 33-61); 
and creating a list of suggested tests from the determined probabilities (e.g. 
Col. 10, line 66-Col.11, line 35). 
The reiection of claim 4 

In response to Appellants' argument that "Booth et al. do not contain any 
teaching or suggestion determining a probability of a correct diagnosis and a 
probability of an incorrect diagnosis for the test suite using the database; and 
creating a list of suggested tests from the determined probabilities" (Appeal 
Brief, page 12, paragraph 3), the examiner disagrees. Booth et al. teach 
determining a probability of a correct diagnosis (e.g. Col.6, lines 49-54) and a 
probability of an incorrect diagnosis for the test suite using the database (e.g. 
Col.9, lines 33-61); and creating a list of suggested tests from the determined 
probabilities (e.g. Col. 10, line 66-Col.l 1 , line 35), each suggested test on the list 
comprises test coverage (e.g. Col.1 1 , lines 29-31 ). 

The reiection of claim 6 

In response to Appellants' argument that "Booth et al. do not contain any 
teaching or suggestion evaluating a diagnostic efficacy of the test suite using a 
probability of one or both of a correct diagnosis and inconrect diagnosis by the 
test suite; and identifying a test to delete from the test suite, the deletable test 
having a minimal effect on an overall diagnostic efficacy of the test suite" 
(Appeal Brief, page 13, paragraph 2), the examiner disagrees. Booth et al. teach 
evaluating a diagnostic efficacy of the test suite (e.g. Col.9, lines 14-15) using a 
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probability of one or both of a correct diagnosis and incorrect diagnosis (failure 
probability term) by the test suite (e.g. a probability of detectability by the test 
suite, Col.1 1, lines 15-18, lines 24-29, Col. 13, line 64); and identifying a test to 
delete from the test suite, the deletable test having a minimal effect on an overall 
diagnostic efficacy of the test suite (e.g., if an incorrect diagnosis is made, the 
automated analysis system identifies ways of changing the rank order of 
diagnoses, including coverages that can be reduced and identification of 
operation violations that can be eliminated, Abstract). 
The reiection of claim 1 1 

In response to Appellants' argument that "Booth et al. do not contain any 
teaching or suggestion creating a simulation database of test suite; determining a 
probability of one or both of a correct diagnosis and incorrect diagnosis for the 
test suite using database" (Appeal Brief, page 14, paragraph 6). the examiner 
disagrees. Booth et al. teach the test suite analysis 116 and model debug 
analysis 120 can be used with simulated data 124 (Col.6, lines 52-54). Either 
simulated data (FIG. 1, 124) or historical TFC data (FIG. 1, 126) may be used for 
such analysis. If failure distribution information is available, the simulated failures 
may be created accordingly (Col. 9, lines 4-8). Although Booth et al. do not 
specifically disclose the claimed creating a simulation database of test suite, this 
feature is seen to be an inherent teaching of that step since the test suite 
analysis 116 can be used with simulated data 124 (Col.6, lines 52-54). Booth et 
al. disclose the simulated data (FIG. 1 , 124) may be used for such analysis. If 
failure distribution information is available, the simulated failures may be created 
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accordingly (Col.9, lines 4-8) that some type of creating a simulation database of 
test suite must be present for determining probability of one or both a correct 
diagnosis and an incorrect diagnosis for the test suite as intended. Booth et al. 
teach step of determining a probability of one or both a correct diagnosis and an 
incorrect diagnosis for the test suite using the database (e.g. Col.1 1, lines 15-18, 
lines 27-29); using the determined a probability to evaluate the test suite (e.g. 
Col.9, lines 14-15). 

The rejection of claim 32 

In response to Appellants' argument that "Booth et al. do not contain any 
teaching or suggestion a computer program stored in the memory and executed 
by the processor, wherein the computer program comprises instructions that, 
when executed by the processor, implement evaluating the test suite using a 
probability of one or both a correct diagnosis and an incorrect diagnosis by the 
test suite to determine the efficacy." (Appeal Brief, page 27, paragraph 3), the 
examiner disagrees. Booth et al. teach the test system 100, the model based 
diagnostic system 106, test suite analysis 116 and model debug 120 are all 
typically computer or processor based. That is, software is executed by a 
processor within an automated test system or the test system is controlled by a 
separate computer. In general, the test suite analysis 116 and model debug 120 
can be performed by a processor or computer used for the test system 100 or 
used for the model based diagnostic system 106. Alternatively, test suite 
analysis and model debug may be peri'ormed offline on a separate system or 
distributed among several systems. Note also that some of the statistical 
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inferences involved may be determined by fuzzy logic or neural networks so that 
the word "automated" is not restricted to sequential execution of computer 
instructions (Fig.1, Col.6, line 60-Col.7, line 7). The test suite may be evaluated 
for overall accuracy by analysis of historical data (FIG. 1 , 126). For example, the 
system could compute the mutual information (or other statistical or information 
theoretic measures) between the model-based diagnosis and the distribution of 
TFC's recorded in the database (CoL9, lines 14-19). Booth et al. disclose a 
method of identifying improvements to a test suite in diagnosability analysis, the 
test suite based on a model of a system, the method comprising automatically 
identifying components, the failure of which has a probability of detectability by 
the test suite (Col.8, lines 15-31. Col. 13, lines 59-65). The debug of the model 
based on incorrect diagnoses (Col.9, lines 33-34). Historical data may be used in 
conjunction with the model-based diagnostic system: providing cases for 
performance evaluation of the model-based system; allowing evaluation of the 
effect of changes suggested by debugging on overall performance (Col.1 1 , lines 
50-60). This feature is seen to be an inherent teaching a computer program 
stored in the memory and executed by the processor, wherein the computer 
program comprises instructions that, when executed by the processor, implement 
evaluating the test suite using a probability of incorrect diagnosis by the test suite 
to determine the efficacy as intended. 
The rejection of claim 33 

In response to Appellants' argument that "Booth et al. do not contain any 
teaching or suggestion the instructions that evaluate the test suite comprise one 
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or both of suggesting a test to add to the test suite, and identifying a test to 
delete from the test suite" (Appeal Brief, page 8, paragraph 2), the examiner 
disagrees. Booth et al. disclose an automated analysis system that identifies 
detectability problems, diagnosability problems, and possible ways to change 
rank order of diagnoseis in a diagnostic system and makes the problems and 
possible improvements visible to test programmers to aid in test improvement. 
Components that have no coverage and components that have inadequate 
coverage (according to a heuristic criteria) are identified as potential detectability 
problems. Components that are exercised by identical operations in all tests are 
identified as diagnosability problems. If an incorrect diagnosis is made, the 
automated analysis system identifies failing tests that have no coverage of any 
component in the true failure cause. In addition, if an incorrect diagnosis is 
made, the automated analysis system identifies ways of changing the rank order 
of diagnoses, including coverages that can be reduced and identification of 
operation violations that can be eliminated or deliberately added (Abstract). This 
feature is seen to be an inherent teaching the instructions that evaluate the test 
suite comprise one or both of suggesting a test to add to the test suite, and 
identifying a test to delete from the test suite as intended. 
The reiection of claim 34 

In response to Appellants' argument that "Booth et al. do not contain any 
teaching or suggestion creating a simulation database of the test suite; 
determining a probability of one or both of a correct diagnosis and an incorrect 
diagnosis using the database; and using the determined probability to evaluate 
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the test suite." (Appeal Brief, page 17, paragraph 4, page 18, paragraphs 3^), 
the examiner disagrees. Booth et al. teach the test suite analysis 116 and model 
debug analysis 120 can be used with simulated data 124 (Col.6, lines 52-54). 
Either simulated data (FIG. 1, 124) or historical TFC data (FIG. 1, 126) may be 
used for such analysis. If failure distribution information is available, the 
simulated failures may be created accordingly (CoL9, lines 4-8). Although Booth 
et al. do not specifically disclose the claimed creating a simulation database of 
test suite, this feature is seen to be an inherent teaching of that step since the 
test suite analysis 116 can be used with simulated data 124 (Col.6, lines 52-'54). 
Booth et al. disclose the simulated data (FIG. 1 , 124) may be used for such 
analysis. If failure distribution information is available, the simulated failures may 
be created accordingly (Col. 9, lines 4-8) that some type of creating a simulation 
database of test suite must be present for determining probability of one or both 
a correct diagnosis and an incorrect diagnosis for the test suite as intended. 
Booth et al. teach step of determining a probability of one or both a correct 
diagnosis and an incorrect diagnosis for the test suite using the database (e.g. 
Col.1 1, lines 15-18, lines 27-29); using the determined a probability to evaluate 
the test suite (e.g. Col. 9, lines 14-15). 
The rejection of claim 35 

In response to Appellants' argument that "Booth et al. do not contain any 
teaching or suggestion using the determined probability of both a correct 
diagnosis and an incorrect diagnosis comprises creating a list of suggested tests 
to add to the test suite, each suggested test having an associated test coverage" 
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(Appeal Brief, page 8, paragraph 2), the examiner disagrees. Booth et al. teach 
determining a probability of a correct diagnosis (e.g. Col. 6, lines 49-54) and a 
probability of an incorrect diagnosis for the test suite using the database (e.g. 
Col. 9, lines 33-61); each suggested test having an associated test coverage (e.g. 
Col. 11, lines 29-31). 

B. The appellants have argued, regarding the 35 U.S.C. 102(b) rejection 
of claims 5, 7, 10, and 36 as anticipated by Booth et al. (US 5,922,079), 
regarding the 35 U.S.C. 103(a) rejection of claims 8 and 31 as being 
unpatentable over Booth et al. (USP 5,922,079) in view of Kanevsky et al. (USP 
6,167,352), and regarding the 35 U.S.C. 103(a) rejection of claim 38 as being 
unpatentable over Booth et al, (USP 5,922,079) in view of Preist et al. (USP 
5,808,919). The examiner agrees with the arguments. Therefore, the rejection of 
claims 5, 7, 8, 10, 31 , and 38 under Booth, Kanevsky et al.,and Preist et al. are , 
withdraw. 

CONCLUSION: 

For the above reasons, it is believed that the rejections should be 
sustained. 

An appeal conference responsive to the reply brief was held on 
09/29/2005 with Drew Dunn, SPE of AU 2872, and John E. Barlow, SPE of AU 
2863. 

Respectfully Submitted, 



Application/Control Number: 10/053,748 
Art Unit: 2863 



Page 19 



John H. Le 



September,29, 2005 




John I 

(ipeivlsoryt^rtentEMinlner 
Technology CentBr280O 



AGILENT TECHNOLOGIES, INC. John E. Barlow 4^ 

Legal Department, M/S DL429 Conference 
Intellectual Property Administration 
P.O. Box 7599 
Loveland, CO 80537-0599 



